Skip to content

feat: Implement delta migration support for content mapping and entry…#1090

Merged
umesh-more-cstk merged 4 commits into
devfrom
feature/delta-contenttypescreen
Jun 15, 2026
Merged

feat: Implement delta migration support for content mapping and entry…#1090
umesh-more-cstk merged 4 commits into
devfrom
feature/delta-contenttypescreen

Conversation

@yashin4112

Copy link
Copy Markdown
Contributor

🔗 Jira Ticket

CMG-997


📋 PR Type

  • ✨ Feature
  • 🐛 Bug Fix
  • 🔥 Hotfix
  • ♻️ Refactor
  • 🧹 Chore / Dependency Update
  • 📝 Documentation

📝 Description

What changed?

New "Map Entry" step (delta migration, iteration 2+)

  • Split the migration flow into 6 steps from iteration 2 onwards: Legacy CMS → Destination Stack → Map Content Fields → Map Entry → Test Migration → Execute. Iteration 1 stays the original 5-step flow (no Map Entry).
  • Step 3 (Map Content Fields) now shows only new content types (not migrated in the previous iteration) — re-mapping already-migrated types risks data loss.
  • Step 4 (Map Entry) is a new standalone screen that shows only already-migrated content types that have entry mappings, and lets the user select entries to update.

Single-route, filter-based content type fetch

  • getContentTypes takes a filter query param (new | old) instead of a new endpoint. On iteration 2+ it diffs the current vs previous iteration's contentTypesMapper.json by otherCmsUid to return the new-only or old-only slice. Iteration 1 is untouched.
  • Map Entry list excludes content types with no entryMapping, and derives each content type's status icon (green "Updated" / blue "Mapped") from the saved isUpdate state so it persists correctly across reloads.

Iteration-aware step numbering

  • Backend current_step state machine is now iteration-aware (DELTA_STEPPER_STEPS adds Map Entry as step 4, shifting Testing → 5, Execute → 6 on delta iterations).
  • Frontend stepper, CTA button states, step gates (project_current_step), and the side-nav (migrationSteps.json) all resolve step numbers by iteration.

Stepper / navigation fixes

  • Step completion is driven by the persisted current_step, so navigating back to review a completed step no longer resets the progress bar, and completed steps stay navigable.
  • Restart resets the progress bar correctly; the Start Migration button is disabled via the actual API call (not the completion flag) so live logs still stream.
  • Friendlier empty state on delta iterations ("No new content types — you can still continue") with the Continue button enabled and the "Go to Legacy CMS" action hidden.

Bug fix — entry update data loss

  • removeEntriesFromDatabase previously deleted every entry present in the entry mapper, even ones never migrated (no contentstackEntryUid), silently dropping them. Now it only removes entries that already exist in Contentstack (have a contentstackEntryUid); isUpdate ones are collected for update, the rest are left to be created fresh.

Tests

  • Added unit tests for the updateEntryCli service (basic-auth & OAuth paths, missing stack key, auth errors, stdout/stderr log-level classification, non-zero exit handling, warning mapping).

Why?

Delta (re-)migrations need to map and update entries of already-migrated content types across iterations without losing data or re-mapping content that was already migrated. The previous flow had no Map Entry step, no iteration-aware step numbering, and a bug that dropped never-migrated entries during update.


🧩 Affected Areas

  • api — Node.js backend
  • ui — React frontend
  • upload-api — Upload API server
  • docker / docker-compose
  • CI / GitHub Actions workflows
  • Environment variables / config
  • Other:

🧪 How to Test

Iteration 1 (fresh migration):

  1. Run a migration end to end — confirm the flow has 5 steps (no Map Entry).
  2. On Map Content Fields, confirm all content types show; empty list shows the error state with Continue disabled.

Iteration 2+ (delta / restart):

  1. Restart a completed migration (iteration bumps) and confirm the flow now has 6 steps with Map Entry as step 4.
  2. Step 3 shows only NEW content types; if none, the friendly "No new content types" state shows with Continue enabled.
  3. Step 4 (Map Entry) shows only already-migrated content types that have entries; select entries and click Save — the content type icon turns green.
  4. Reload the page — content type icons keep the correct green/blue status without needing to open each one.
  5. Walk all 6 steps forward/back — the progress bar doesn't reset, completed steps are navigable, and current_step in the DB reaches 6 on Execute.
  6. Start a migration — live logs stream and the Start button is disabled while running.

Entry update data-loss fix:
7. Have an entry that was NOT migrated in iteration 1 — confirm it gets created this iteration (not silently dropped). Entries marked isUpdate with a Contentstack uid get updated.

Tests:
8. cd api && npm testupdateEntryCli.service.test.ts passes.

Expected result: Iteration 1 = 5 steps unchanged; iteration 2+ = 6 steps with correct new/old content type splitting, persistent status icons, no progress-bar resets, and no entry data loss.


📸 Screenshots / Recordings

Before After

🔗 Related PRs / Dependencies


✅ Author Checklist

  • Branch follows naming convention: feature/, bugfix/, or hotfix/ + 5–30 lowercase chars
  • Jira ticket linked above
  • Self-reviewed the diff — no debug logs, commented-out code, or TODOs left in
  • .env / example.env updated if new environment variables were added — N/A (no new env vars)
  • No sensitive credentials or secrets committed
  • Existing tests pass locally (npm test)
  • New tests written — added updateEntryCli.service.test.ts
  • README.md / docs updated if behaviour changed — N/A
  • Talisman pre-push scan passes (no secrets flagged)

👀 Reviewer Notes

  • The new/old content type split is driven by a single getContentTypes?filter=new|old param + a per-iteration diff on contentTypesMapper.json — no new endpoint.
  • Step numbering is iteration-aware in two places that must stay in sync: createStepper() (UI) and migrationSteps.json all_steps (side-nav), plus the backend STEPPER_STEPS / DELTA_STEPPER_STEPS.
  • Please double-check the entry-update.utils.ts delete rule — entries are removed from import data only when they have a contentstackEntryUid (exist in CS); never-migrated entries are intentionally kept to be created.

Migration v2 · Docs · Issues

… mapping steps

- Added logic to handle delta iterations in the ContentMapper component, allowing for different behaviors based on the iteration number.
- Updated the MigrationFlowHeader to adjust step IDs and button states based on the current iteration.
- Enhanced the HorizontalStepper to maintain step completion status across iterations.
- Introduced new constants for empty state messages in the content mapper.
- Modified API calls to differentiate between new and old content types based on the iteration.
- Map Entry step lists only content types that have entry mappings, with status icons derived from saved isUpdate state.
- Made backend current_step iteration-aware (Map Entry inserts an extra step from iteration 2+).
- Fixed entry-update: only remove entries that already exist in Contentstack (have a contentstackEntryUid); update them when marked isUpdate, otherwise create fresh — prevents data loss of never-migrated entries.
- Updated the LoadUploadFile component to support MySQL connection details editing.
- Adjusted the Migration page to conditionally render steps based on the iteration.
- Improved the handling of empty states in the UI for better user experience during migration.
@yashin4112 yashin4112 requested a review from a team as a code owner June 13, 2026 06:01
@github-actions

Copy link
Copy Markdown

🔒 Security Scan Results

ℹ️ Note: Only vulnerabilities with available fixes (upgrades or patches) are counted toward thresholds.

Check Type Count (with fixes) Without fixes Threshold Result
🔴 Critical Severity 0 0 10 ✅ Passed
🟠 High Severity 14 338 25 ✅ Passed
🟡 Medium Severity 16 2 500 ✅ Passed
🔵 Low Severity 0 0 1000 ✅ Passed

⏱️ SLA Breach Summary

⚠️ Warning: The following vulnerabilities have exceeded their SLA thresholds (days since publication).

Severity Breaches (with fixes) Breaches (no fixes) SLA Threshold (with/no fixes) Status
🔴 Critical 0 0 15 / 30 days ✅ Passed
🟠 High 4 0 30 / 120 days ❌ Failed
🟡 Medium 0 2 90 / 365 days ⚠️ Warning
🔵 Low 0 0 180 / 365 days ✅ Passed

🟠 High Severity - SLA Breached Issues (with fixes)

Showing 4 issue(s) that have exceeded the 30-day SLA threshold:

  1. Interpretation Conflict

    • ID: SNYK-JS-FASTURI-16642394
    • Package: fast-uri@3.1.0
    • Published: 31 days ago (SLA: 30 days)
    • CVSS Score: 8.7
    • CVE: CVE-2026-6322
  2. Directory Traversal

    • ID: SNYK-JS-FASTURI-16642399
    • Package: fast-uri@3.1.0
    • Published: 31 days ago (SLA: 30 days)
    • CVSS Score: 8.7
    • CVE: CVE-2026-6321
  3. Interpretation Conflict

    • ID: SNYK-JS-FASTURI-16642394
    • Package: fast-uri@3.1.0
    • Published: 31 days ago (SLA: 30 days)
    • CVSS Score: 8.7
    • CVE: CVE-2026-6322
  4. Directory Traversal

    • ID: SNYK-JS-FASTURI-16642399
    • Package: fast-uri@3.1.0
    • Published: 31 days ago (SLA: 30 days)
    • CVSS Score: 8.7
    • CVE: CVE-2026-6321

ℹ️ Vulnerabilities Without Available Fixes (Informational Only)

The following vulnerabilities were detected but do not have fixes available (no upgrade or patch). These are excluded from failure thresholds:

  • Critical without fixes: 0
  • High without fixes: 338
  • Medium without fixes: 2
  • Low without fixes: 0

❌ BUILD FAILED - Security checks failed

Please review and fix the security vulnerabilities before merging.

@github-actions

Copy link
Copy Markdown

🔒 Security Scan Results

ℹ️ Note: Only vulnerabilities with available fixes (upgrades or patches) are counted toward thresholds.

Check Type Count (with fixes) Without fixes Threshold Result
🔴 Critical Severity 0 0 10 ✅ Passed
🟠 High Severity 10 338 25 ✅ Passed
🟡 Medium Severity 16 2 500 ✅ Passed
🔵 Low Severity 0 0 1000 ✅ Passed

⏱️ SLA Breach Summary

⚠️ Warning: The following vulnerabilities have exceeded their SLA thresholds (days since publication).

Severity Breaches (with fixes) Breaches (no fixes) SLA Threshold (with/no fixes) Status
🔴 Critical 0 0 15 / 30 days ✅ Passed
🟠 High 0 0 30 / 120 days ✅ Passed
🟡 Medium 0 2 90 / 365 days ⚠️ Warning
🔵 Low 0 0 180 / 365 days ✅ Passed

ℹ️ Vulnerabilities Without Available Fixes (Informational Only)

The following vulnerabilities were detected but do not have fixes available (no upgrade or patch). These are excluded from failure thresholds:

  • Critical without fixes: 0
  • High without fixes: 338
  • Medium without fixes: 2
  • Low without fixes: 0

⚠️ BUILD PASSED WITH WARNINGS - SLA breaches detected for issues without available fixes

Consider reviewing these vulnerabilities when fixes become available.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds iteration-aware “delta migration” behavior across the backend + UI, including a new Map Entry step (iteration 2+) and safer entry update handling to avoid data loss during re-migrations.

Changes:

  • Introduces iteration-aware step numbering (5-step iteration 1 vs 6-step delta iterations) across UI stepper/side-nav and backend state machine.
  • Implements filter-based content-type fetching (filter=new|old) to split “new” vs “already-migrated” content types across iterations, and adds the new Map Entry UI step.
  • Fixes entry update logic to only remove entries from import data when they already exist in Contentstack, and adds unit tests for updateEntryCli.

Reviewed changes

Copilot reviewed 21 out of 22 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
ui/src/utilities/constants.ts Adds Content Mapper empty-state copy for delta vs iteration 1.
ui/src/services/api/service.interface.ts Adds iteration to migration response typing.
ui/src/services/api/migration.service.ts Adds filter query param support for content type fetch.
ui/src/pages/Migration/index.tsx Adds iteration-aware step list (Map Entry step) and CTA routing/disable fixes.
ui/src/context/app/app.interface.ts Adds hasNoContentTypes flag for step-3 Continue gating.
ui/src/components/Stepper/HorizontalStepper/HorizontalStepper.tsx Changes completion tracking to use persisted current_step.
ui/src/components/MigrationFlowHeader/index.tsx Makes header CTAs/disable logic iteration-aware and adds step-3 empty-state gating.
ui/src/components/LegacyCms/Actions/LoadUploadFile.tsx Adds tooltips to edit icons.
ui/src/components/ContentMapper/index.tsx Fetches only “new” content types for step 3; improves delta empty state and step gating.
ui/src/components/ContentMapper/index.scss Adds centered delta empty-state styling and formatting cleanup.
ui/src/components/ContentMapper/entryMapper.tsx Refactors EntryMapper into standalone delta step (step 4) with CT list + entry table.
ui/src/cmsData/migrationSteps.json Adds Map Entry step definitions and renumbers later steps.
package.json Adds fast-uri dependency constraint.
package-lock.json Updates fast-uri lockfile version/resolution.
api/tests/unit/services/updateEntryCli.service.test.ts Adds unit tests for updateEntryCli (auth paths, errors, logging, exit codes).
api/src/utils/entry-update.utils.ts Fixes entry-removal/update selection to prevent dropping never-migrated entries.
api/src/utils/entry-update-script.cjs Removes a debug console.info.
api/src/services/runCli.service.ts Uses iteration-aware final step number when marking migration complete.
api/src/services/projects.service.ts Makes current-step state machine iteration-aware and adds MAP_ENTRY transition.
api/src/services/migration.service.ts Makes Testing step update iteration-aware.
api/src/services/contentMapper.service.ts Adds iteration diffing + `filter=new
api/src/constants/index.ts Adds DELTA_STEPPER_STEPS + getStepperSteps(iteration) helper.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread ui/src/components/ContentMapper/index.tsx
Comment thread ui/src/utilities/constants.ts Outdated
Comment thread ui/src/components/ContentMapper/entryMapper.tsx
Comment thread ui/src/components/ContentMapper/entryMapper.tsx
Comment thread ui/src/pages/Migration/index.tsx Outdated
@github-actions

Copy link
Copy Markdown

🔒 Security Scan Results

ℹ️ Note: Only vulnerabilities with available fixes (upgrades or patches) are counted toward thresholds.

Check Type Count (with fixes) Without fixes Threshold Result
🔴 Critical Severity 0 0 10 ✅ Passed
🟠 High Severity 10 338 25 ✅ Passed
🟡 Medium Severity 26 2 500 ✅ Passed
🔵 Low Severity 0 0 1000 ✅ Passed

⏱️ SLA Breach Summary

⚠️ Warning: The following vulnerabilities have exceeded their SLA thresholds (days since publication).

Severity Breaches (with fixes) Breaches (no fixes) SLA Threshold (with/no fixes) Status
🔴 Critical 0 0 15 / 30 days ✅ Passed
🟠 High 0 0 30 / 120 days ✅ Passed
🟡 Medium 0 2 90 / 365 days ⚠️ Warning
🔵 Low 0 0 180 / 365 days ✅ Passed

ℹ️ Vulnerabilities Without Available Fixes (Informational Only)

The following vulnerabilities were detected but do not have fixes available (no upgrade or patch). These are excluded from failure thresholds:

  • Critical without fixes: 0
  • High without fixes: 338
  • Medium without fixes: 2
  • Low without fixes: 0

⚠️ BUILD PASSED WITH WARNINGS - SLA breaches detected for issues without available fixes

Consider reviewing these vulnerabilities when fixes become available.

@umesh-more-cstk umesh-more-cstk merged commit 4f92fc0 into dev Jun 15, 2026
9 checks passed
@umesh-more-cstk umesh-more-cstk deleted the feature/delta-contenttypescreen branch June 15, 2026 07:57
chetan-contentstack pushed a commit that referenced this pull request Jun 16, 2026
…creen

feat: Implement delta migration support for content mapping and entry…
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants